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The behavior of mobile devices is highly non deterministic and barely predictable due to the inter¬ 
action of the user with its applications. In consequence, analyzing the correctness of applications 
running on a smartphone involves dealing with the complexity of its environment. In this paper, 
we propose the use of model-based testing to describe the potential behaviors of users interacting 
with mobile applications. These behaviors are modeled by composing specially-designed state ma¬ 
chines. These composed state machines can be exhaustively explored using a model checking tool 
to automatically generate all possible user interactions. Each generated trace model checker can be 
interpreted as a test case to drive a runtime analysis of actual applications. We have implemented a 
tool that follows the proposed methodology to analyze ANDROID devices using the model checker 
SPIN as the exhaustive generator of test cases. 


1 Introduction 

At present, smartphone technology is ubiquitous and changes constantly. Users use their mobiles not only 
as phones, but as compact computers, able to concurrently provide services which are rapidly created, 
updated, renewed and distributed. In this scenario of continuous evolution, different operating systems 
have been developed such as SYMBIAM, lOS, WINDOWS PHONE and ANDROID, which allow phones to 
support more and more complex applications. These platforms define new models of execution, quite 
different from those used by non-mobile devices. For instance, one of the most defining characferisfics 
of fhese sysfems is fheir open and evenf-driven nafure. Mobile devices execute a confinuous cycle fhaf 
consisfs of firsf, wailing for Ihe user inpul and second, producing a response according lo lhal inpul. In 
addilion, Ihe infernal slruclure of mobile systems is conslrucled from a complex combination of applica¬ 
tions, which enable users lo easily navigate Ihrough Ihem. Thus, allhough, al a lower level, Ihe execution 
of applications on a mobile device involves Ihe concurrenl execution of several processes (for inslance, 
in ANDROID, applications are JAVA processes executing on Ihe underlying LINUX operating system), Ihe 
way Ihese applications inleracl wilh each olher and wilh Ihe environmenl does nol correspond wilh Ihe 
slandard interleaving based concurrency model. 

If is clear lhal Ihe execution of applications on Ihese new operating systems, such as ANDROID |T|, 
may lead lo Ihe appearance of undesirable bugs which may cause Ihe phone lo malfunction. For example, 
mobile devices may display Ihe typical errors of concurrenl systems such as violations of safety and 
liveness properties. However, Ihere are olher bugs inherenl lo Ihe particular concurrency model supported 
by Ihe new platforms which are nol direclly analyzable using currenl verification technologies. For 
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example, applications could incorrectly implement the life cycles of their activities or services (in the case 
of ANDROID), or may misbehave upon the arrival of unexpected external events. In addition, conversion 
errors, unhandled exceptions, errors of incompatibility API and l/O interaction errors as described in lfT6l 
may also appear. 

Different techniques for analyzing the execution of mobile platforms have been proposed. Ver¬ 
ification approaches such as model checking ||9l can be applied to the software for mobile devices 
Model checking is based on a exhaustive generation of all the inter leavings for the thread¬ 
s/processes. A major problem to apply this technique to the real code, like mobile applications, it the 
need to construct a model of the underlaying operating system or libraries ifTOi rTHTTI. The open nature of 
these platforms, which are continuously interacting with an unspecified environmenf, makes ofher anal¬ 
ysis techniques such as testing, monitoring, and runtime verification more suifable fo check bugs wifhouf 
loo much exlra efforl fo model fhe operaling sysfem or fhe libraries. There have been several recenl 
proposals l[T^l24l[T8t for lesling in Ibis framework wifh commercial fools mill. In fhese approaches, 
fesl cases are randomly generated wifh fools such as Monkey and MonkeyRunner ||T]. 

Testing and runfime verification maybe also combined, as described in [til, fo consfrucf verification 
fools for mobile applicafions Il20ll2ti]l . On fhe one hand, fhe careful seleclion of fesl cases guides fhe ex¬ 
ecution of fhe device, while, on fhe ofher, fhe runfime verificalion module implemenfs observers devoted 
fo analyzing fhe Iraces produced by fhe device. The runfime verificalion module was already addressed 
by some aulhors of Ihis paper in ifldll . Here we focus on describing how fhe generalion of fesl cases may 
be carried oul following fhe model-based testing approach Il25]l supported by model checking fools. 

Our proposal is based on fhe idea lhaf allhough fhe inferaclion belween fhe user and fhe mobile 
device is complelely undetermined, each applicafion is associaled wifh a sel of intended user behaviors 
which define fhe common ways of using fhe applicafion. For each applicafion, or more precisely, for each 
applicafion view, we use slate machines fo consfrucf a non deterministic model representing fhe expected 
use of fhe view/applicafion. This slate machine is buill semiaulomalically, wifh informalion provided by 
fhe experl (fhe app designer or fester) and by ANDROID supporling fools like UIautomatorViewer. 
Then, all fhese view models may be convenienlly composed fo consfrucf a non deterministic model of 
fhe user inferaclion wifh a subsel of mobile applications of inleresl. Due fo fhe way of building fhe slate 
machines, each execution Irace of fhe composed state machine corresponds fo a possible realistic use of 
fhe mobile. Thus, fhe generation of fesl cases is reduced fo fhe generation of all possible behaviors of 
fhe composed machine, which may be carried ouf by a model checking fool. Alfhough fhe mefhodology 
proposed does nol depend on fhe underlying mobile operaling sysfem, fhe fool has been builf on fhe 
assumplion lhaf fhe operaling system is ANDROID. 

The paper provides Iwo main conlribufions. The firsl one is fhe formal definilion of a special lype 
of slate machine lhaf models fhe expecfed user inferaclion wifh fhe mobile applicafion. The approach 
fo modeling is completely modular in fhe sense lhaf adding (or removing) new view slate machines fo 
incorporate (eliminate) user behaviors does nol affecl fhe resl of slate machines lhaf have already been 
defined. The second one is a melhod fo employ fhe explicif model checker SPIN |[T5l lhaf lakes fhe 
composed slale machine as inpul and and produces a significanl sel of fesl cases lhaf generate Iraces for 
runfime verificalion fools. We have conslrucfed a fool chain which implemenfs bofh modeling and fesl 
generation phases fo shows fhe feasibilily of fhe approach in pracfice. 

The resl of fhe paper is organized as follows. Section describes our approach fo using model 
checking for fesl case generation. Section inlroduces fhe lesling plalform lhaf we are developing. 
Section provides a formal description of fhe behaviour of composed slate machines. Section uses 
well known ANDROID applications fo describe how our approach for fesl case generalion is implemenled. 
Seclionj^gives a comparison wifh relaled work. Lasl secfion summarizes conclusions and fulure work. 
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mtype = i state_init, state_l, state_2 }; 

typedef Device { byte transitions [MAX_TR]; short index; bool finish; } 

Device devices [DEVICES] ; 
mtype state [DEVICES] ; 

active proctype traceCloser() provided (devices[DEVA].finish && devices[DEVB].finish) { 
end_tc: outputTransitions() 

} 

active proctype device_A() { 
state[DEVA] = state_init; 
do 

:: state [DEVA] == state_init -> transition(DEVA, BUTT0N_1); state [DEVA] = state_l 

:: state [DEVA] == state_l -> transition(DEVA, SWIPE); state [DEVA] = state_l 

:: state [DEVA] == state_l -> transition(DEVA, BUTT0N_2); state [DEVA] = state_2 
:: state [DEVA] == state_2 -> transition(DEVA, MESSAGE); break 

:: state [DEVA] == state_2 -> transition(DEVA, BACK); break 

od ; 

devices [DEVA] .finish = true; 

} 

active proctype device_B() { 
state[DEVB] = state_init; 

devices [DEVB] .finish = true; 

} 

Listing 1: Sample PROMELA specification for test generation 


2 Model checking for test case generation 


SPIN ifTSl is a model checker that can be used to verify the correctness of concurrent software systems 
modeled using the specification language PROMELA. The focus of the tool is on the design and validation 
of computer protocols, although it has been applied to other areas. SPIN can check the occurrence of 
a property over all possible executions of a system specification, and provide counterexamples when 
violations are found. 

We use the SPIN model checker in our approach for automatically generating test cases from appli¬ 
cation models in the following way. First, each device will be represented by a single PROMELA process, 
which contains a state machine that models the applications contained on that device. The state machines 
themselves are written as loops, where each branch corresponds to a transition triggered by an event. The 
current state of each state machine (stored as a global PROMELA variable) determines which branches 
are active and may be taken. The right hand side of each branch records the transition and updates the 
current state. This PROMELA specification is explored exhaustively by SPIN in order to generate all pos¬ 
sible test cases described by the application model, taking all possible alternatives when there is more 
than one active branch at the same time. 

Listing [T] shows an example of a PROMELA specification that follows the approach outlined above. 
This example contains two devices and with their corresponding state machine (device_A() in line[^ 
and device_B() in line 19), with two states plus the initial state. The transition function is used to 
record the user or system transition associated with each branch. In order to complete a test case, all 
devices must have finished fheir respecfive sfafe machines (lines [TT] and [2^, usually when fhe do loop is 
exiled (lines [T4| and [T5]). This enables fhe traceCloser process lo be execufed due fo ifs schedulabilily 
resfriclions (line[^, which prinls fhe Iransilions of fhe generated lesl case. 

In addition lo fhe currenl slate (line[^, Ihis PROMELA specificalion also keeps a lisl of fhe Iransilions 
laken on fhe lesl currenlly being generaled (line|^. The purpose of Ihis dala slruclure is Iwofold. On fhe 
one hand, outputTransitions will prinl fhe Irace sfored here. On fhe olher hand, fhe hislory of fhe 
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Figure 1: Architecture 


current trace is kept inside the Spin’s global state, which is taken into consideration when deciding if a 
state has already been visited. Thus, the same transition may be taken more than once if possible (e.g. 
line [T^ , since the history of the states will be different. However, this requires the maximum depth of 
exploration to be bounded by the MAX_TR constant (line|^. 


3 Architecture of the platform 

Figure [T] shows the general structure of tools that combine testing and runtime verification techniques to 
analyze the behavior of applications running on mobile devices. The bottom side uses observers/monitors 
to analyze the resulting execution traces and verify whether they comply with the expected properties 
as implemented in the tool DRAGONFLY ifldl [T3l . The top side of the figure shows fhe generation of 
fesf cases considered in fhis paper. The Tesfer is fhe experf responsible for modeling fhe behavior of fhe 
applicafions fo be analyzed using a sfafe charl diagram. These models may be consfrucfed as par! of fhe 
design phase of fhe applicafions, and are characferized by fheir composifional nafure: funcfionalify can 
be added fo an exisfing view wifhouf essentially alfering fhe exisfing behavior. 

Figure]^ shows fhe complefe process of our acfual proposal for fesf generation and execution, which 
is divided info fhree main modules: 

• Modeling. UIautomatorViewer fool from android fools exfracfs fhe confrols definition in 
each view of fhe ANDROID applicafion under analysis. Then, fhe confrols definifion and fhe sfafe 
chart diagrams are associafed info a Model. xml file wifh a given sfrucfure. 

• Test Case Generation. Creafes a fesf case generafor per XML file model info a PROMELA file. 
The SPIN model checker ifTSll performs an exhausfive search of all valid pafhs in fhe model using 
fhe mefhod explained in Secfion]^ which correspond fo fesf cases, and generafes an XML file for 
each one wifh fhe appropriafe sequence of user inpuf evenfs. 

• Test Case Execution. Generafes each fesf class provided using fhe valid pafhs described info XML 
file by fhe fesf case generafing module. Then, fhey can be execufed by fhe ANDROID framework, 
and sends fhem fo fhe devices fo be execufed using fhe UlAUTOMATOR fool which is an exfension 
of JUnif fool using fo write user interfaces fesf cases for ANDROID. 

The following sections provide defails abouf fhe infernal behavior of fhe Modeling and Tesf Case 
Generator modules, which are fhe aim of fhis work. 












Espada, Gallardo, Salmeron & Merino 


11 



Figure 2: Test generation and execution process 


4 Formal Description of models 

In the following description, we define the behaviour of mobile applications through the composition 
of state machines at different abstraction levels. The lowest level is composed of view state machines. 
A view corresponds to a mobile screen, with its buttons, text fields, etc., through which users may 
interact with the device. When the view is active, users may fire events through its interface. A view 
state machine models the possible behaviors of the user when he/she is making use of the view. These 
behaviors coincide with the sequence of events fired by the user. Sometimes one of these events makes 
a different view becomes active. We have modeled this control transfer between views through the 
composition relation of view state machines from which device state machines are constructed. Device 
state machines use the connection states to switch from the current active view to a different view. In 
this formalization, the specific applicafions fo which each view belongs have not been taken into account, 
that is, we only model the transfer from one view to another, irrespective of whether both views belong to 
the same application. In the sequel, we use symbols —> / —to denote the transition relation of the view 
state machines M/M,-. In addition, symbol —defines the binary relation which allows us to connect 
view state machines. Finally, represents the transition relation of the device state machine which is 
constructed from relations —> / —and —>c- 

Since ANDROID applications are event driven, we may consider that each test case corresponds to 
the sequence of events fired which drive the mobile behaviour. In the formal description, events are the 
labels of transitions ^d) and have the natural meaning. For instance, s s' means that 

event e must be fired fo be able to transit from 5 to 5 '. 

4.1 View state machines 

Definition 1 A view state machine is a tuple M = (£,/,—> ,E,C,F), where £ is a finite set of states, 
/ C £ are the initial states, C C £ are the so-called connection states, F is the set of final states, E is 
the set of user events, and —)-C £ x £" xF is the labelled transition relation. Sets I, C and F are mutually 
disjoint. 

Final states are states from which it is not possible to evolve. Connection states are states from which 
it is possible to transit a different state machine. These states are essential to model the switch between 
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typical views of smart phone devices. Usually, when a new view is called, the execution of the system is 
supposed to return to the view caller. To take this behavior into account, we assume that each connection 
state s £C has a related state return{s) G £ which represents the state to be returned when the new view 
invoked from s has finished its execution. 

We partition set of events E into two disjunct sets: the set of user events, denoted as £"+, which 
contains events such pressing a button, etc., and the set of system events, denoted as , which includes, 
for instance, events corresponding to system responses to user requests. In the following, we use e^ 
to represent user events and system events, and we use e to refer to events which may be of any of both 
types. 

View state machines are deterministic in the sense that if 5 —> ^i, and s S 2 and e = e , then = S 2 

that is, the machine defines, at most, a transition for each pair state/input event. 

We now define the notion of flow (an execution in a view state machine), and the test cases generated 
from flows. 

Definition 2 Given a view state machine M = (£,/, —>,E,C,E), we define the set Elow{M) = {^o ^ 
^ f,Sn G F UC} of all sequences of states, allowed by M, starting at an initial state of 
M, and ending at a final or connection state ofM. The length of a flow is the number of its states. Given a 
flow of length n, (j) = sq s,, G Elow{M), the sequence of events determined by (j) (the test case) is 

test{<l)) = ei . e„. We define the set of test cases allowed by M as TC{M) = {test{^)\^ G Elow{M)}. 

According with Definition]^ test cases are finite sequences of user and system events. For instance, 
sequence e^ -e^ -ef ■ e^ represents a test case where the user first fires events e^ and ei^, then the system 
fires e^, and finally user fires ej). Thus, user and system events are similarly dealt with during the 
generation of test cases. The difference between them is of importance when test cases are transformed 
into code to be executed on the mobile as described in Section |5l User events will be transformed into 
non-blocking calls to methods that simulate the real occurrence of the event, while system events will 
correspond to calls to blocking methods which wait for the arrival of the system event. 

4.2 Composition of view state machines 

In this section, we describe how view state machines are composed to construct flows that navigate 
through different views representing realistic ways of using a mobile. 

We first define the transition between different view state machines. This transition is realized 
through the binary relation defined between the connection and initial states. The idea is as fol¬ 
lows. Assume that the flow in execution belongs to a view state machine M,, and that a connection state 
cs of Mi has been reached. If relation Sf defines a transition from cs to some initial state of other machine 
Mj, the flow could jump from M, to Mj, and proceed following the transition relation of Mj. This jump 
implies the change in the activity visible in the device from M,- to My. In the sequel, we call active the 
view state machine which is visible in the device, and create to the rest of view state machines which 
have been created but are not currently visible in the device. 

Given a finite family of state machines M,- = we define £ = U”^j£/, 1 = 

E = C = and E = UjLjF)-. In addition, we denote with SUE the set of call events that 

provoke the switch between active view state machines. 

Definition 3 Let us assume a finite family of state machines, Mj = (£/,/;, —>i,Ei,Ci,Ei). The connection 
of view state machines Mi , • • • ,M„ is given by a binary relation , • • • ,Mn) C C x S’ x I, that con¬ 
nects connection states with initial states. In the following, we denote 3-uples {si,e,Sj) of - ■ ■ ,M„) 

as Si Ac Sj. Observe that source and target machines i and j may coincide. 
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When a new view is created, the call event may specify some parameters that determine how it must 
be started or finished. For instance, if the view has already been created, the caller may choose whether 
to reuse the previously created view or, to the contrary, create a new one. Additionally, when the new 
created view has finished fhe execution, fhe caller view may aufomafically become active or nof. Boolean 
functions reuse,auto .return : S' —)■ {false,true} esfablish fhese parameters for fhe call evenfs. Alfhough 
fhere are ofher paramefers fhaf can be defined in fhe call evenfs, fhese fwo are sufficienf fo describe fhe 
mobile behaviour. 

We now define fhe device state machine which composes fhe behavior displayed by fhe view slate 
machines using fhe conneclion relalion. 

Definition 4 Let us assume a finite family of state machines, Mi = —t-i,Ei,Ci,Fi), and a connection 
relation of Mi ,■■■ ,Mn, 3f{Mi ,■■■ ,Mn), as defined above. The device slate machine 


^ = Mi\\\---\\\M„\\\fif{Mi,---,M„) 


is defined as the state machine (£ x £* x S’* ,1, —t-d,E,F) where 

1. £* is the set of finite sequences of states ofL, and S* is the set of finite sequences of call events. 

2. Transition relation —is defined by the rules below. 

We call configurations fhe slates of device slate machines. A configuration is a 3-uple {s,h,eh) where 
s is fhe currenl slate of fhe acfive view stale machine, i.e., fhe view visible in fhe mobile. Sequence h is 
fhe slack of stales si-S 2 ---s„ which conslilule fhe hislory of fhe view machines which have been created 
(and have nof been yef deslroyed) in fhe device buf which are nof currenlly visible. Each slale Si of 
‘5'i • ‘S '2 • • - is a conneclion slate of a view slate machine which was acfive, buf a fransilion from Sj lo 
anofher view machine look place, and fhe view slate machine of Si became inactive. Finally eh = e\ - ■ ■ Cn 
is fhe history of evenfs lhal have provoked a view swilch in fhe currenl execution. Thus, e,- ^ S is, fhe 
evenl which fired fhe fransilion from slate Si fo an initial slale of anofher view slale machine. In fhe 
following, £ represenfs fhe empty (even!) history. 

The evolufion of configurations is given by fhe fransilion relation —defined by fhe rules in Figure]^ 
Relalion is conslrucled from fhe fransilion relalions of view slale machines —and fhe binary 

conneclion relalion —In fhese rules, given a history of slates . Sn and fhe index j of a view slate 

machine Mj, function top : L* x JV —)• ZU {_L} relums fhe Iasi slate of Ihe view slate machine My in 

Ihe sequence . Sn. Thai is, top{s\ . s„,i) relurns 5^., if 1 < k < n is Ihe biggesl index such lhal 

Sk G Zy, or _L, if such a slate does nol exist 

Rule R1 slates lhal a fransilion inside a view slate machine M, corresponds to a fransilion in Ihe 
device slate machine. Rules R2, R3 model a Iransilion from machine M; to machine My when bolh Ihe 
new slate s' and Ihe evenl e are added to Ihe slate and evenl histories of Ihe currenl system configuration. 
Rule R2 is applied when evenl e does nol involve reusing a previously created view {reuse{e) is false), 
while R3 applies when a view of My, should have been reused {reuse{e) is Irue), bul Ihe currenl slate 
history does nol conlain one {top{s\ ■ ■ -Sn,]) = -L). Rule R4 defines a fransilion from machine M, to My 
by reusing a previously created flow of My (reuse{e) is Irue) which is stored in Ihe configuration history 
{top{si ■ ■ -Sn,]) = sf). Finally, R5 defines Ihe case when Ihe flow of Ihe currenl active view has finished, 
and Ihe execution musl continue wilh Ihe view stored al Ihe top of Ihe slate history. Olherwise, lhal is, if 
auto.return{e) relurns false, Ihe currenl configuration {s,h,eh) cannol evolve. 
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Rl. 


s —s 


R2. 


5 G Ci,s -^c s',^reuse{e) 


R3. 


R4. 


R5. 


{s,h,eh) {s',h,eh) {s,h,eh) — {s',h- return{s),eh ■ e) 

s G Ci,s' G Ij,s Ac s',reuse{e),top{si ■ --Sn,]) = _L 
{s,h,eh) Arf {s',h- return{s),eh ■ e) 

s G Ci,s' G Ij,s Ac s' ,reuse{e),top{si ■ --Sf,,]) = Sk 
• ‘ ‘ Sn^€\ ' ' ^ d {skAt * ‘ ‘ Sk—lt^l * ‘ ‘ ^k—l) 

s G Fi, auto-ret urn{e) 


R6. 


{s,h-s',eh-e) — {s',h,eh) 

e+ 

Co —>d Cl 


R7. 


Cq -At/' Cj , ^ dh 


{co,c'Q,dh) -^d\\d’ {c\,c'Q,dh + {e+}) {co,c'o,dh) —^d\\d' {coA^dh - {e+}) 


Figure 3: Transition relation rules 


Definition 5 Given a device state machine 


= (Ixl* x^*,/, ArfAU(f,F) 


1. the trace-based semantics determined by 3) is given by the set of finite/infinite sequences 

of configurations (flows) produced by the transition relation —^dffom an initial state, that is, 

= {(5o,e,£) Arf (5i,/ii,e/zi)--Ao e A- 

2. Given a flow 0 = cq ^d ci ^d C 2 • • • G the test case determined by 0 is the sequence of 

events test{(^) = e\-e 2 --- 

3. The set o/test cases determined by a set of flows ST is TC{^) = {test{t)\t G ^}. 


Thus, a flow (j) G consists of a sequence of view state machine flows (Definitionconnected 
throw connection states. Flow (j) may finish at a final state of some view state machine, or may be 
infinite. The length \(p\ of a flow (j) is the number of its states (configurations), if it is finite, or oo, 
otherwise. Given a flow (/> = cq ci C 2 • • • G G{Si), we define the truncated flow of n, (/)”, as (p 
iff 101 <= n or 0” = Co ^d ci A^ C 2 • • • c„_i, otherwise. Considering this, we define the set of 

traces as the set all traces of truncated up to length n, that is, = {0"|0 G 

Observe that the state space of device state machines is not finite because configurations include 
the state and event histories which may have arbitrary lengths. In addition, the state space generated 
when an explicit model checker is constructing all the flows allowed by a device state machine is non- 
finite not only due to the state and event histories, but also because the matching algorithm, carried out 
during the state space search, must take into account both the current state of the flow and the history 

ef , et 

of the previous states of the flow. This allows that, for instance, flows 0i = {so,£,s) —>d {siAA) —^d 


{s 2 ,e,e) Arf {s 3 ,e,e) and 02 = Ao,£,e) ( 54 ,£,£) A^ Ai,e,e) A^ {s 2 ,e,e) A^ {s 3 ,e,e} can be 

both generated by the model checker although when constructing 02 state 5 i has been already visited as 
explained in Section]^ 
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In consequence, the models of device state machines are not, in general, state finite which means 
that, the model checking process does not, in general, terminate. In the current implementation, we have 
solved this problem by bounding the depth of the execution flows analyzed generating for some 

fixed n. 

4.2.1 Composing several devices 

The exfension of fhe sfafe machine model fo several devices is carried ouf by composing fhe device sfafe 
machines by inferleaving. Thus, if co c\ and c'q -4-^/ c\ are a fransifions in devices ^ and 3i', respec¬ 
tively, fhen fhey allow fhe fwo following fransifions, (co,Cq) -^djld' (ci,Cq) and (co,Cq} -^d\\d' (co,Cj) 
in fhe inferleaved composifion of & and The communication befween bofh devices is modeled by 
a user even! in fhe sender device (fhe device fhaf sfarfs fhe communication), and a sysfem evenf in fhe 
receiver device (fhe device fhaf expecfs fhe message). 

This is described in fhe lasf fwo rules of Figure Rule R6 handles fhe fransifion from fhe sender, 
and R7 handles fhe fransifion in fhe receiver. Nofe fhaf dh denofes fhe sef of sysfem evenfs produced buf 
nof yef consumed. Thus, for insfance, using fhe previous example, if ei = is an even! fhaf implies a 
communication from & fo and e\ = is fhe corresponding even! fo be read by from we would 
generate fhe fesf cases and ■ ef. Nofe fhaf in fhe second fesf case, fhe mefhod fhaf implemenfs 

fhe fransifion for fhe receiver evenf will suspend fhe execufion of unfil evenf ef is fired by 

In addition, when dealing wifh more fhaf one device, we make use of model checking optimiza¬ 
tion fechniques such as parfial order reduction iflSl fo avoid fhe generation of differenf fesf cases fhaf 
correspond fo a single feasible inferacfion befween fhe devices. 

5 Case Study 

In Ibis secfion, we describe how fhe behavior of mobile applications is modeled and how fesfs cases are 
aufomafically generated from fhese models. 

5.1 Modeling 

We firsf need fo consfrucf an absfracl model of fhe system fo be analyzed, using sfafecharfs and following 
fhe nofions of view and device sfafe machines given in Secfion|^ This model should include fhe relevanf 
user inferacfions for fhe fesfs we wanf fo perform. For insfance, a fesf case which is affecfed by whefher 
fhe GPS is on or off may include user inferacfions fo change ifs sfafus, while ofher fesfs may nof need 
fhose inferacfions. A fesf case generafor will be creafed from fhis model using automatic fransformafions. 
This modeling sfep can be performed separafely from fhe design of fhe applicafion, or in combination 
as is cusfom in ofher model-driven tools such as IBM Rafional Rhapsody lO ■ In addifion, fhe confrols 
of each screen have to be exfracfed and modeled, so fhaf fransifions on fhe sfafe machines can be fied fo 
acfions performed on fhese confrols. 

We use a scenario wifh fwo applicafions, Facebook and YouTube. This scenario is composed of fhree 
views {HomeView, CommentView and MovieView), which describe fhe behavior of a user placing a link 
to a YouTube video in a Facebook commenf, and wafching fhis or ofher videos in YouTube. The sfafe 
machines can be modeled using UML as shown in Figure These sfafe machines include additional 
information required fo correlate fhem wifh fhe applicafions and fheir views. An XML definifion of fhe 
model can fhen be aufomafically generafed from fhese sfafe machines. Lisfingj^shows part of fhis XML 
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Figure 4: Facebook and YouTube model 


Likel You like this. 



Figure 5: Identifying control groups 


definition In particular, it contains the state machine associated with the Home view of the Facebook 
app. Each state machine may define several states and transitions. In addition to simple transitions 
between states of the same state machine, it is also possible to define transitions that call another state 
machine and, upon its termination, continue in the caller machine. The type and through attributes 
identify the type of transition and the state machine to call (if any). Listing [^provides examples of both 
simple (line [T0| ) and complex (lines and [TT| ) transitions. Each transition has an unique ID within its 
view that is used to identify transitions, and also declares the user or system event that triggers it. 

The events that fire the transitions in Eigure are the user actions performed on controls placed 
in visible views. We organize controls into group of controls according to the actions associated with 
each. Eigure shows some of the control groups that have been identified in the CommentView View. 
Eor instance, the Comment group could represent any of the text fields to write a comment, and the 
clickYouTubeLink identifies links to YouTube videos. 

These control groups are declared in a control definition file with the help of the UlAUTOMA- 

*More complete versions of this an other parts of the model are available online at http: //morse .uma. es 
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<Application naine = "Facebook" pack age = " com.facebook. android " > 

<Views > 

<View name="HomeView" controlsFile="Home.xml" > 

<StateMachines > 

<StateMachine name="HomeUpdate"> 

<States><State name="SO"/><State name="SI"/></States> 

<Transitions > 

<Transition ID="1" event="Swipe" prev="" next="S0" type="Simple"/> 
<Transition ID="2" event="Comment" prev="S0" next="S0" 
through="CommentView" type="View"/> 

<Transition ID="3" event="Swipe" prev="S0" next="Sl" type="Simple"/> 
<Transition ID="4" event="ClickYouTubeLink" prev="S0" next="S0" 
through="ViewingMovieStateMachine" type="StateMachine"/> 

<Transition ID="5" event="Swipe" prev="Sl" next="Sl" type="Simple"/> 
<Transition ID="6" event="Comment" prev="Sl" next="S0" through="CommentView 
type="View"/> 

<Transition ID="7" event="Swipe" prev="Sl" next="" type="Simple"/> 

Listing 2: State machine configuration 


<node index="0" text="" testGroup="" .... 

<node index="0" .... 

<node testGroup="clicLike" IsFixedValue="" PatternOrValue="" index="0" text="Like" 
resource-id="id/feed_feedback_like_container" clickable="true" 
long-clickable="false" password="false" ... /> 

Listing 3: Control group definition 


TORViewer tool im. This tool analyzes each view without requiring its source code, and generates 
an UIX (XML) file confaining fhe hierarchy of confrols in fhe view. Lisfing shows par! of fhe gener¬ 
ated file for fhe Facebook application. The affribufes associafed wifh each confrol in fhe UIX file include 
fhe kind of actions fhaf fhe confrol supporfs, such as clickable or scrollable. The UIX file is fhen 
cusfomized fo bring fogefher fhe confrols which belong fo fhe same group by selling fhe controlGroup 
allribufe. Some confrols accepl parameters which may also be included as attributes in this file. For 
insfance, fhe values inlroduced in lexl fields may be fixed or generaled aulomafically according fo some 
pattern. 

5.2 Test case generation 

We are now ready fo generale fhe corresponding lesl cases in an exhauslive manner. The XML file is 
aulomafically Iransformed info a PROMELA specification that follows the same principles described in 
Section]^ but with a few additions to acomodate the structure of ANDROID applications. Each device is 
still represented by a single process, but their state machines are defined in separale Mines, one per each 
app, view and state machine, which can then be composed. In addition, there may be device-specific app 
and view inlines, since views and slafe machines can be assigned fo a parlicular device. In fhe simplesl 
form of composilion, device processes call fheir app inlines, app inlines call Iheir view inlines, and view 
inlines call fheir slafe machine inlines. On fhe olher hand, a slate machine may call anolher view or slafe 
machine. When fhis happens, fhe slafe of fhe previous slafe machine should be stored such lhal when 
fhe new one is finished, fhe slafe is correclly restored. To supporl fhis we inlroduce a backstack dafa 
slruclure, where fhe slafe of fhe currenl stale machine is always al fhe fop of fhe slack. 

Lisfing shows a simplified exlracf of fhe PROMELA specificafion generated for fhe Facebook and 
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typedef Backstack { mtype states[MAX_BK]; short index; } 

Backstack backstacks[DEVICES]; 

#define currentBackstack devices[device].backstack 

#define currentState currentBackstack.states[currentBackstack.index] 

active proctype device_219dcac41() { 
if 

true -> app_219dcac41_Facebook(D_219dcac41); 
true -> app_219dcac41_YouTube(D_219dcac41); 
f i ; 

devices[D_219dcac41].finished = true 

} 

inline statemachine_Facebook_HoineView_HomeUpdate(device) { 
currentState = State_Facebook_HoineView_HoineUpdate_init ; 
pushToBackstack(device, State_Facebook_HoineView_HomeUpdate_init); 
do 

:: currentState == State_Facebook_HoineView_HomeUpdate_SO -> 
transition(device, VIEW_HomeView, 2); 
view_Faceb 00 k_CommentView(device); 

currentState = State_Facebook_HoineView_HomeUpdate_SO 
:: currentState == State_Facebook_HoineView_HomeUpdate_SO -> 
transition(device, VIEW_HomeView, 4); 

statemachine_YouTube_MovieView_ViewingMovieStateMachine(device); 
currentState = State_Facebook_HoineView_HomeUpdate_SO 


od ; 

popFromBackstack(device) 


Listing 4: PROMELA specification for Facebook and YouTube 


YouTube example. The backstack data structure is shown on line A new element is pushed to or 
popped from the backstack at the beginning or end of a state machine, respectively (lines and 26 1 , 
while currentState always point to the top of this stack for each device. 

Each sequence of transitions generated by SPIN is translated into a UiAutomatorTestCase subclass, 
where each transition is implemented by a method. This class simulates the actions performed by the 
user, such as pressing buttons or swiping. The code shown in Listing]^ shows part of a test case obtained 
from the model of Figure]^ in particular a user adding a link to a YouTube video in a Facebook comment, 
and later watching that video on the YouTube application. The file is compiled info a . dex (ANDROID 
applicafion binary) file, and fhen deployed info a ANDROID device and execufed using fhe adb fool. 

Table [T] provides some quanfifalive resulfs of the number of test cases generated and the computa¬ 
tional effort required, for several scenarios, averaged over three runs. Device 219dcac4 was assigned 
only the Facebook application, while device 219dcac41 was assigned both, although in both cases other 
modeled applications may be reached from the assigned ones. The fourth column declares the maximum 
depth allowed for the test case transitions generated for a device. The fifth column represents the total 
time spent to generate the test cases from the XMF models. The last three columns are stats taken from 
the SPIN execution, namely the number of SPIN states generated, the size of each state, and the total 
memory spent, respectively. These results show how adding the YouTube application, which is fairly 
isolated, has little impact in the results (rows 3 and 4 of data). 


6 Comparison with related work 

There are other proposals to apply model-based testing to ANDROID applications. Some of them consider 
that the testing process starts without a precise model of the expected behavior of the applications and 
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public class TestDevicel extends UiAutomatorTestCase { 

// Transition 2 previous SO next SO on view HomeView 

public void TestFacebookCoinment2() throws UiObjectNotFoundException { 

UiObject control = new UiObject(new UiSelector(). 

className("android.widget.TextView") .index (1) .textContains("Comment")); 
control.click(); 

} 

// Transition previous SO next SO on view HomeView 

public void TestFacebookclicYouTubeLink27() throws UiObjectNotFoundException { 
UiObject control = new UiObject(new 

UiSelector () .className("android.view.View") .index(3)); 
control.click(); 

} 

// Transition 1: previous next YO on view MovieView 

public void TestYouTubeplaypause28() throws UiObjectNotFoundException { 

UiObject control = new UiObject(new 

UiSelector () .className("android.view.View") .index(4)); 
control.click(); 

} 

} 


Listing 5: Generated UiAutomatorTestCase 


Devices 

Configuration 

Results 

219dcac4 

219dcac41 

Backs tack 

Transitions 

# Test Cases 

Time (s) 

# States 

State Size (B) 

Memory (MB) 

/ 


4 

20 

5641 

1.0 

307234 

84 

156.8 

/ 


4 

26 

111317 

9.0 

6063398 

92 

728.6 


/ 

4 

20 

5660 

1.0 

307493 

84 

156.8 


/ 

4 

26 

111342 

9.0 

6063735 

92 

728.6 

/ 

/ 

4 

10 

1872 

7.0 

4039337 

100 

560.3 

/ 

/ 

4 

12 

12180 

52.3 

28972472 

108 

3445.2 


Table 1: Test case generation results 


they focus on techniques to obtain such model. MobiGUITAR framework iQ automatically construct 
a state machine of one application by executing events in the running application and recording a tree 
with fireable events for each new state. The authors use a ’’breadth-first” traversal of the apps GUI for 
open source applications. As far as they are not considering any knowledge on the way of using the 
application but they are making an exhaustive execution, they need some criteria to assume whether 
some states are equivalent to prevent state explosion. The Swift-Hand technique proposed in |8] employs 
machine learning to construct an approximated model of the application during the testing process. Their 
aim is to cover as much behavior as possible, making the execution to enter in unexplored parts of the 
state space. In our method, we separate test generation from testing and the states in our high-level state 
machines are limited and differentiated by design. So our models are more compact, and for instance, 
compared with MobiGUITAR we do not need extra work to remove unrealistic test cases. In addition, 
our approach allows to generate test cases for several applications that interact using ANDROID intents, 
while the complexity of the runtime based modeling process for MobiGUITAR and Swift-Hand makes 
them more suitable for single applications. 

Like in our proposal, other works also consider the existence of a formal specification of the appli¬ 
cations to start the test generation.In ifTTll the authors describe how to follow a property-driven method 
build the models in Alloy, a formal language based on high order logic. In their proposal the role of the 
model checker in our approach is done they the Alloy analyzer, that generates positive (expected) and 
negative (undesired) test cases. Like in our approach, they use XML based transformations to translate 
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the test cases to some executable form in order to activate the applications under test. Apart from the 
inside technologies (model checking vs constraint solver), the main difference in both proposals is the 
way to obtain the refined executable model. The Alloy specification in IfTTl is constructed manually, 
while the PROMELA specification in our work is done automatically from the high level design of the 
view state machines. We still need to work in the same case study to get a quantitative comparison on 
the human and computational effort required in both approaches. 

There are other model-based testing tools for Android which are not focused on models that consider 
the user inputs. For instance, the tool APSET Il23]l considers manually constructed formal models of 
vulnerability patterns to generate test cases for ANDROID applications. Test generation is implemented 
with an ad-hoc algorithm that also considers the compiled code of the application and the configuration 
files in fhe ANDROID system. 


7 Conclusions and Future Work 

ANDROID systems have a complex architecture designed to support the concurrent execution of appli¬ 
cations on devices with limited resources. Here we have presented a model-based testing approach for 
generating test cases for ANDROID applications, which takes into account the way in which these appli¬ 
cations interact with the user and with each other. We model the expected user behavior by composing 
state machines, and then explore this model exhaustively with SPIN to obtain all possible user behaviors, 
which correspond to test cases. These test cases are then executed in the device simulating the user 
inputs. In contrast with other approaches that generate random input events, our approach produces re¬ 
alistic user behaviors. Although our tool is currently geared towards ANDROID, the same principles can 
be applied to analyze applications in other mobile platforms, such as lOS and WINDOWS Mobile. 

The next step of our work will be to connect the generated test cases with a runtime verification 
monitor DRAGONFLY ifT^ [T3i . In addition, we are working in adding more runtime information, like 
energy consumption, to perform richer analysis. 
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